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ABSTRACT 


Faced with aging aircraft and fewer acquisitions. Naval Aviation has redoubled its 
effort to preserve assets through preventative maintenance and reduction of aircraft 
mishaps. Eighty percent of all mishaps are due in part to human error, and approximately 
one out of five major mishaps are due to maintainer, line, or facility related factors. 
Among various efforts to systematically reduce mishaps is the use of the Maintenance 
Climate Assessment Survey (MCAS). This survey is designed to capture maintainer 
perceptions of safety. This thesis analyzes the results of 977 responses to MCAS given to 
the 3 rd Marine Air Wing (MAW) maintenance personnel. In addition, it explores the 
MCAS’s relationship with human errors present in 21 maintenance-related incidents 
(MRIs) using the Human Factors Analysis and Classification System - Maintenance 
Extension. This analysis finds statistically different responses among the squadrons of 
the 3 rd MAW to the MCAS. These differences show the MCAS can detect variations 
between aviation units and associated Model of Organization Safety Effectiveness 
components. While no significant correlation between the nine adequately surveyed 
squadrons and their MRIs is found, a content analysis of the MCAS shows there is a 
relationship between the MRIs a squadron experiences and the items of the six MOSE 
components. 
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EXECUTIVE SUMMARY 


Faced with aging aircraft and fewer acquisitions. Naval Aviation has 
redoubled its efforts to preserve assets through preventative maintenance and reduction of 
aircraft mishaps. While dramatic strides were achieved in the last 50 years in reducing 
the Class A Flight Mishap (FM) rate, the presence of human error has remained relatively 
constant. In 1996, a Human Quality Management Board (HFQMB) was established to 
address human errors in FMs, and initially targeted aircrew error mishaps. The HFQMB 
achieved positive results, but as the last decade came to a close it was clear maintainer 
error would have to be targeted in order preserve aviation assets. 

The Naval Safety Center and the School of Aviation Safety at the Naval 
Postgraduate School are addressing aircraft maintenance issues in order to achieve 
additional Class A FM reductions. Their efforts have generated two tools to assess 
maintainer error and trends and factors contributing to an incident, including the safety 
climate within which the organization operates. Schmidt, Schmorrow and Hardee (1998) 
develop the Human Factors Analysis Classification System - Maintenance Extension 
(HFACS-ME) taxonomy used to classify causal factors that contribute to maintenance- 
related incidents (MRIs). Schmorrow (1998), investigates all Naval Aviation class A, B 
and C MRIs and Teeters (1999) looks at all MRIs form the Fleet Logistic Support Wing. 
Both use HFACS-ME to analyze past mishap data building a Poisson model to predict 
future mishaps and through analysis of potential reductions associated with these error 
types forecast a decline in MRIs resulting in significant cost savings. Baker (1998), 
working with a model of safety effectiveness based on the concept of High Reliability 
Organizations, uses the existing Model of Safety Effectiveness and the Command Safety 
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Assessment (CSA) to develop a 35-item Maintenance Climate Assessment Survey 
(MCAS). Further work by Goodrum (1999), Oneto (1999), School of Aviation Safety, 
and AIRPAC refined the MCAS into a 43-question survey. 

This study examines the MCAS ability to capture maintainer perceptions of 
maintenance operations and explores the relationship between MCAS and aircraft 
maintenance-related incidents (MRIs) using HFACS-ME. With the study of 977 
maintainer surveys from the 3 rd MAW, an effort to assess the maintenance safety climate 
within the wing is undertaken. Commander 3 rd MAW, realizing the School of Aviation 
Safety was working on reducing MRIs, decided to seek assistance to access the reasons 
behind an increasing mishap rate in Fiscal Year 1999. The School of Aviation Safety 
provided various training and analysis including: CSA, MCAS, Ground Crew 
Coordination Training (GCT), Maintenance Operational Risk Management (MORM) 
awareness briefs, the HFACS-ME examination of Class A, B, C and Hazard Report 
(HAZREP) MRIs, and a package on Class A Helicopter and tactical aircraft (TACAIR) 
mishaps. 

Through the use of Principal Component Analysis, this study validates the use of 
the MCAS in assessing a squadron’s maintenance safety climate. Using Analysis of 
Variance (ANOVA) and Multiple Comparison testing, this study shows that there is a 
statistically significant difference between squadrons. Tukey’s Multiple Comparison is 
used to aggregate units into above average, average and below average maintenance 
safety climate. Linear Regression is then preformed using mean MCAS responses as the 
predictor variable and the MRI rate as the response variable. While no significant 
correlation between the nine adequately surveyed squadrons and their MRIs is found, a 
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content analysis of the MCAS shows there is a correlation between the MRIs a squadron 
experiences and the items of the six MOSE components. 
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I. 


INTRODUCTION 


“Aviation in itself is not inherently dangerous. But to an even greater degree than 
the sea, it is terribly unforgiving of any carelessness, incapacity, or neglect” 
(Anonymous). Risk is inherent in Naval Aviation, however the success of Naval 
Aviation is based on a willingness to balance risk with the opportunity to take decisive 
actions necessary to succeed in battle (DON, 1997). However, squadron commanders 
have a fundamental responsibility to safeguard personnel and material resources, and to 
accept only minimal risk to accomplish an assigned mission (DON, 1997). There have 
been several major accidents in which organizational error plays a major role. One need 
only recall Three Mile Island (March 1979), the Space Shuttle Challenger (January 
1986), and Chernobyl (April 1986) as examples of major accidents in which 
organizational error plays a significant role (Redmill & Rajan, 1997). Roberts (1990) 
observes that some organizations operate in a complex and hazardous environment, yet 
experience a very low accident rate. She terms these activities “high reliability 
organizations” or “HROs.” Ciavarelli (1997) highlights research showing that HROs 
have certain key characteristics in common such as leadership style, management 
policies, procedures standardization, superior training, safety reward, adequacy of 
resources and staffing, and effective risk and hazard management. In a study of nuclear 
powered aircraft carriers, Roberts (1990) formally recognizes Naval Aviation as an HRO. 

Despite Naval Aviation’s success in reducing by 50 percent its Class A (those 
resulting in death, permanent disability, or loss of $1 million or more) Flight Mishap 
(FM) rate each successive decade between Fiscal Years (FY) 1950 and 1990, the 
proportion of aircraft losses attributable to human error has remained relatively constant 


1 


this decade (Ciavarelli & Figlock, 1997). Following the 1996 crash of a Navy F-14 in 
Nashville, Tennessee a Human Factors Quality Management Board (HFQMB) was 
formed with a goal to reduce Naval Aviation’s Class A FM rate due to human error by 50 
percent. The HFQMB initially focused on aircrew error, which was found to be a casual 
factor in 60 percent of all mishaps. However, it has increased its focus to encompass 
maintenance error that comprises approximately 15-20 percent of Class A mishaps. 

A. BACKGROUND 

The 3 rd Marine Air Wing (MAW) has experienced a dramatic rise in their FY 
1999 Class A Flight Mishap (FM) rate and, intrigued by the efforts of the HFQMB, 
requests from the School of Aviation Safety at the Naval Postgraduate school support in 
cutting their FM rate. The mission of the 3 rd MAW is to provide combat-ready 
expeditionary aviation forces capable of short-notice worldwide deployment to Marine 
Air Ground Task Force (MAGTF) fleet and unified commander. The main aircraft of 3 rd 
MAW are the AH-1W “Super Cobra”, UH-1N “Huey”, CH-53 “Super Stallion”, CH-46E 
“Sea Knight”, F/A-18D “Night Attack Hornet”, F/A-18 “Hornet”, AV-8B “Harrier” and 
the C-130T “Hercules”. The 3 rd MAW is composed of 25 combat squadrons, three 
training squadrons and four Marine Aviation Logistics Squadrons (MALS) divided 
among four Marine Air Groups (MAGs) located on three different Marine Corps Air 
Stations in Southern California. The MALS provide intermediate maintenance and 
aviation logistics support to squadrons within each respective MAG, whereas each 
squadron performs it’s own organizational maintenance. 

The function of the MAGs is to perform various related aviation tasks for the 
Marine Corps. The primary mission of MAG-11 (F/A-18, F/A-18D & C-130T) is to 
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provide air support to Marine Air Ground Task Force (MAGTF) commanders. MAG-13 
(AV-8B) provides close-air support, conducts armed reconnaissance, and assumes limited 
air-defense roles. MAG-16 (CH-53 & CH-46E) performs its assigned mission of 
transportation and re-supply for Marine air and ground units. Finally, MAG-39 (AH-1W 
& UH-1N) provides utility helicopter support, close-in fire support, fire support 
coordination, aerial reconnaissance, observation and forward air control in aerial and 
ground escort operations during ship-to-shore movement and subsequent operations 
ashore. 

The 3 rd MAW, in dealing with is recent increased mishap rate, received 
constructive support from the School of Aviation Safety at the Naval Postgraduate 
School, in the form of training and analysis including: Command Safety Assessment 
(CS A), Maintenance Climate Assessment Survey (MCAS), Ground Crew Coordination 
Training (GCT), Maintenance Operational Risk Management (MORM) awareness briefs, 
the Human Factors Analysis and Classification System - Maintenance Extension 
(HFACS-ME) examination of Class A, B, C and Hazard Report (HAZREP) maintenance- 
related incidents (MRIs), and a package on Class A Helicopter and tactical aircraft 
(TACAIR) mishaps. This support was implemented to evaluate their maintenance safety 
climate in an effort to understand the causes of the ground-based incidents. 

B. PURPOSE 

The intent of this study is three fold: 1) obtain 3 rd MAW maintainer perception of 
maintenance operations safety with MCAS, 2) determine the if human error is a 
significant factor in 3 rd MAW MRIs using HFACS-ME, and 3) evaluate the ability of the 
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MCAS to detect differences in squadron maintenance safety climate and how it is 
associated with their MRI histories. 

C. PROBLEM STATEMENT 

During FYs 1998 and 1999, 3 rd MAW has suffered a dramatic increase in their 
mishap rate. The commander 3 rd MAW, recognizing MRI’s as a part of the problem, 
elected to have the School of Aviation Safety use the MCAS to capture maintainer 
perceptions of 3 rd MAW maintenance operations and HFACS-ME for classifying human 
error that contributed to the 3 rd MAWs MRIs. The 43-question version of the MCAS was 
administered to the 3 rd Marine Air Wing (MAW) during the last half of 1999 in order to 
capture the maintained perception of their safety climate. In this thesis an analysis of 3 rd 
MAWs MRIs from January 1998 to December 1999 is conducted to detect the human 
error present in the incidents as identified by HFACS - ME. The results are evaluated in 
an attempt to correlate these responses with the MRI data over the past 24 months. This 
research involves collection of survey results, generation of descriptive statistics of the 
survey, and employs statistical methods to address the following research questions. 

1. Are there any noteworthy findings with respect to the MCAS? Are there any 
significant HFACS-ME trends observed for the MRIs? 

2. Does the MCAS capture an overall attitude toward aviation maintenance 
safety and are all the questions in each component required to describe the 
majority of the variance of the component? 

3. Does the MCAS detect maintenance safety climate differences between 
squadrons, type aircraft and Marine Air Groups and are these differences 
statistically significant as to cluster the different squadrons into groups? 

4. Do squadrons that cluster below and above average as related to safety climate 
also have a higher and lower average number of mishaps respectively? 
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D. DEFINITIONS 


This thesis uses the following definitions (DON, 1989): 

Naval Aircraft . Refers to U.S. Navy, Naval Reserve, U.S. Marine Corps, and U.S. 

Marine Corps Reserve aircraft. 

Maintenance-Related Incidents (MRIs) 

Mishap . A Naval Aviation mishap is an unforeseen or unplanned event that 
directly involves naval aircraft, which result in $10,000 or greater cumulative 
damage to naval aircraft or personnel. The mishap is further divided into three 
classes based on the amount of damage to the aircraft, property and personnel 
injury. The following are the definitions of the three classes: 

a. Class A . A mishap in which the total cost of property damage (including 
all aircraft damage) is $1,000,000 or greater; or a naval aircraft is 
destroyed or missing; or any fatality or permanent total disability of a 
person occurs with direct involvement of naval aircraft. 

b. Class B . A mishap in which the total cost of property damage (including all 
aircraft damage) is $200,000 or more but less than $1,000,000 and/or a 
permanent partial disability, and or the hospitalization of five or more 
personnel. 

c. Class C . A mishap in which the total cost of property damage (including all 
aircraft damage) is $10,000 or more but less than $200,000 and/or injury 
results in one or more lost workdays. 

Hazard Report (HAZREP1 An incident in which the total cost of property 
damage (including all aircraft damage) is less the $10,000 and/or injury results in 
no lost workdays. 
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Incidence rate . The total number of Class A, B and C mishaps or HAZREPS per 100,000 


flight hours. 

Aircraft Ground Mishap (AGM) . Those mishaps in which no intent for flight existed at 
the time of the mishap and $10,000 or greater aircraft damage or loss of an aircraft, 
and/or property damage occurred. 

Flight Mishap (FM) . Those mishaps in which there is $10,000 or greater aircraft damage 
or loss of aircraft with the existence of intent for flight for the aircraft existed at the 
time of the mishap. 

Flight Related Mishap (FRM) . Those mishaps in which there is no intent for flight at the 
time of the mishap and $10,000 or greater aircraft damage or loss of an aircraft, 
and/or property damage occurs. 

Maintenance Climate Assessment Survey (MCAS) . A 43-question survey used to gain 
insight into the maintenance community’s perception concerning aviation mishaps 
within the Navy and Marine Corps. 

Human Factors Accident Classification System - Maintenance Extension (HFACS-ME) . 
A taxonomic system used to classify causal factors that contribute to maintenance 
related mishaps. 

High Reliability Organization (HR.O) . An organization that operates in a hazardous 
environment, yet produces a very low rate of accidents and incidents by operating 
effectively, safely, and having the characteristics of leadership, sound management 
policies, procedure standardization, adequacy of resources and staffing, a defined system 
for risk management, and other factors. These traits can be divided into six components: 
1) Process Auditing (PA) - an ongoing system to monitor hazards; 2) Reward System 
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(RS) - expected compensation or disciplinary action used to shape behavior; 3) Quality 
Assurance (QA) - policies and procedures for promoting quality performance; 4) Risk 
Management (RM) - organizational risk perception and mitigation; 5) Command and 
Control (CC) - policies and procedures to mitigate risks and 6) Communication / 
Functional Relationships (CF) - an environment where information is freely exchanged, 
quality assurance is seen as a positive influence and maintenance workers are shielded 
from external pressures to complete a task. This sixth trait encompasses a number of 
items specifically relating to maintenance that did not fit in the original traits of HROs. 

E. SCOPE AND LIMITATIONS 

This study uses the MCAS to survey the maintenance safety climate perceptions 
of the maintainers of the 3 rd MAW and to validate the use of the MCAS as a tool to 
reliably asses this climate. A total of 17 California based USMC Active Duty Squadrons 
from the 3 rd MAW participate in the survey. Only Naval Aviation maintenance personnel 
surveys are used in this analysis. Incomplete surveys or surveys with no variation in the 
responses are omitted. 

The survey is administered at the various commands on site and participants are 
chosen from maintainers in the squadron in a non-random order. That is, only those 
maintainers that can participate in the survey at the time of administration are sent to 
participate. The differences in total surveys collected per squadron are due to the 
availability of maintainers and the commands willingness to participate in the survey. 
Also, during the timeframe of this study, all AV-8B “Harriers” are grounded for several 
weeks to assess the reliability of certain components of the airframe. 
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The Naval Aviation Safety Program mandates the investigation and reporting of 
all Class A, B and C incidents (OPNAV 3750.6Q, 1989). However, there are a number 
of difficulties in using the reports of these incidents. The information collected on 
incidents is not in HFACS format nor does it emphasize human factors. Further, fidelity 
is lost by transferring the incident investigation into the Safety Information Management 
System (SIMS) database. In addition, since OPNAV 3750.6Q only recommends but does 
not require the reporting of HAZEPS there number is underreported in the SIMS 
database. Finally, commands can influence the recorded or reported cost associated with 
a mishap thus lowering it from a mandatory reported Class C incident to a non-reported 
HAZREP. 

Chapter II contains a literature review on investigative scope, high reliability 
organizations, human factors quality management board, maintenance climate assessment 
survey development, and human factors - maintenance extension development. In 
Chapter III the methodology used in this study is discussed. Chapter IV presents the 
results of this study. Lastly, Chapter V gives conclusions include findings and 
recommendations. 
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II. LITERATURE REVIEW 


An in depth literature review is given in this chapter to give the reader an 
introduction to the relationship between organizational effects and safety and how two 
tools; the Maintenance Climate Assessment Survey (MCAS) and the Human Factors 
Analysis and Classification System (HFACS) can be used to understand this relationship. 
It will begin with an overview of high-reliability organizations. Next, a history of the 
Human Factors Quality Management Board (HFQMB) is discussed followed by MCAS 
and HFACS-Maintenance Extension (ME) development. The chapter finishes by 
summarizing this body of research and makes recommendations to tie HFQMB, MCAS 
and HFACS-ME into a three-pronged strategy to analyze risk and recommend a course of 
action to prevent a mishap from occurring in aviation organizations. 

A. HIGH RELIABILTITY ORGANIZATIONS 

One factor recognized as significantly contributing to a decrease in aircraft 
accidents is the technology advances in the design and maintenance of modem military 
aircraft (Roberts, 1990). However, Roberts (1990) finds this level of technical 
complexity makes military aircraft vulnerable to accidents or failures that cannot be 
predicted during the design process. In an effort to understand why certain organizations 
having a high level of complexity operating in a hazardous environment do not 
experience greater accidents as complexity increases, she observes that these 
organizations have certain factors in common. She calls these organizations “High 
Reliability Organizations” or “HROs”. 

Technology in general, aids in reducing mishaps, but the existing organizational 
research is inadequate to understand what organizational processes define HROs 
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(Roberts, 1990). Her research shows, that HROs have like leadership style, management 
policies, procedures standardization, superior training, and reward systems. These ideas 
and research by Libuser (1994) lead to the development of a Model of Organizational 
Safety Effectiveness (MOSE). The MOSE model organized HRO traits into the five 
categories of Process Auditing, Reward System, Quality Assurance, Risk Management 
and Command and Control (Libuser, 1994). The five MOSE components are 
summarized as follows: 

1. Process Auditing - a system of ongoing checks to identify a hazardous 
condition and a process to take corrective actions. 

2. Reward System - a system to recognize and reward safe behaviors, and to 
discourage unsafe behaviors. 

3. Quality Assurance - Sets control procedures to monitor quality and correct 
deviations to referenced standards. 

4. Risk Management - Systematic process to identify and manage risk. 

5. Command & Control - an environment where information is freely 
exchanged, quality assurance is seen as a positive influence and maintenance 
workers are shielded from external pressures to complete a task. 

B. HUMAN FACTORS QUALITY MANAGEMENT BOARD 

In an attempt to address human error in Naval Aviation mishaps, a HFQMB was 
established in 1996. Its charter is to analyze human factor involvement in Naval Aviation 
mishaps and in Naval Aviation operations and develop interventions for them. Initially, 
the HFQMBs goal was to cut the mishap rate associated with human error in half 
(Nutwell & Sherman, 1997). The HFQMB used a “three-prong approach”: 1) Mishap 
Data Analysis, 2) Benchmarking and 3) Climate Safety Assessment to identify human 
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error, uncover the best practice and find process improvements, and assess safety climate, 
respectively (See Figure 1). 


Mishap Data 
Analysis 


Benchmarking 


Command Safety 
Assessment 




Figure 1. HFQMB Methodology (Schmidt, 1998) 


Faced with aircrew error contributing to three out of five (Naval Safety Center, " 
1996) Class A Flight Mishaps (FMs) the HFQMB focused initially on them, and it 
consequently resulted in significant reductions in the Class A FM rate. These results 
coupled with emerging requirements to concentrate on aging aircraft related issues 
convinced the HFQMB to expand its scope to include human factors in maintenance- 
related mishaps (MRMs) using the same process (Schmidt, Schmorrow & Hardee, 1998). 

C. MCAS DEVELOPMENT 

Ciavarelli and Figlock (1997) develop a 67-item prototype “Command Safety 
Assessment” (CSA) .for aircrew based on the descriptive MOSE components derived 
primarily from the works of Libuser (1994), Roberts (1990), Zohar (1980). Through the 
survey of 1,254 naval aviators and flight officers they are able to differentiate between 
squadrons having high scores for HRO components and those with low ones. This 
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research represents a first attempt to focus on key organizational issues in order to 
improve our understanding of the possible influence that a command may have in the 
chain of events leading to a maintenance-related aircraft mishap. 

Where Ciavarelli and Figlock focus on organizational effects relating to aircrew 
safety issues, Baker (1998), Goodrum (1999), and Oneto (1999) focus on the 
organizational effects pertaining to maintainer safety issues. Baker (1998) uses the 
existing CSA and MOSE to develop a prototype MCAS. It consists of 15 demographic 
and 67 maintenance-related items. The demographic items include inquiries about rank, 
community, shift worked, total years of service, and total years of aviation maintenance 
experience. The maintenance-related items come from a candidate battery of over 200 
items developed from the CSA, a Naval Safety Center (NAVSAFCEN) questionnaire, 
and noted questions from subject matter experts. Maintenance experts review this initial 
battery of questions and sort them according to the MOSE components. A number of 
items not fitting in the original five categories are placed in a sixth called 
Communication/Functional Relationship (CF) - an environment where information is 
freely exchanged, quality assurance is seen as a positive influence and maintenance 
workers are shielded from external pressures to complete a task. The questions are then 
organized according to their MOSE components. 

Bakers’ (1998) study includes 268 participants from three reserve squadrons that 
represented a spectrum of aviation communities. He tests the reliability and consistency 
of the prototype survey with a squadron from the Fleet Logistic Support (VR) Wing. 
With the survey shown to be content valid. Baker (1998) administers the prototype 
MCAS to two other groups of squadron maintenance personnel from the Maritime Patrol 
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and Reserve Helicopter Wing squadrons located at Moffett FAF and NAS North Island, 
respectively. Studying these Naval Air Reserve Force squadrons as well as input from 
the Naval Safety Center and the School of Aviation Safety his research results in a 35 
item, six-component survey using a Likert type rating scale to assess safety in 
maintenance operations. 

Goodrum (1999) works with VR Wing and surveys nearly 1000 maintainers from 
three different aircraft communities (C-9B, C-20, & C-130T). The results of the study 
concludes that the MCAS developed by Baker (1998) can be utilized as a tool for 
effectively capturing an aviation maintainer’s perceptions of safety in maintenance 
operations. Through analysis of the MCAS responses a number of different areas related 
to the MOSE components are identified as raising concern within the different 
communities. 

Oneto (1999), studying assorted aircraft communities within the Naval Air 
Reserve Force (NARF), also builds upon Bakers (1998) research, surveying 439 
maintenance personnel of the NARF. Oneto’s working independently from Goodrum 
also concludes that the MCAS is an effective tool for capturing an aviation maintainer’s 
perceptions of safety in maintenance operations. However, in his study of NARF 
squadrons, Oneto fails to identify any single MOSE component responsible for 
controlling the outcome of the survey. 

Goodrum and Oneto’s combined studies: 1) validate the stability of the survey 
using Cluster Analysis by achieving similar conclusions through the use of two 
diametrically functioning algorithms and 2) validate the individual survey questions using 
principal component analysis. Further, both studies conclude by proposing a nearly 
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identical 40-item MCAS. Armed with this new revision, the Naval Safety Center, Naval 
Air Forces Pacific and School of Aviation Safety implemented revisions to come up with 
the current 43-item MCAS. 

D. HFACS-ME DEVELOPMENT 

The HFACS is a tool to analyze human errors contributing to Naval Aviation 
mishaps that was developed by Naval Aerospace Psychologists (Wiegman & Shappell, 
1997) of the Naval Safety Center. (Schmidt et al, 1998). HFACS incorporates features of 
Bird’s ‘Domino Theory,’ Edward’s ‘SHEL Model,’ and Reason’s ‘Swiss Cheese Model.’ 
(Schmidt, et al, 1998). Latent conditions and active failures are partitioned into one of 
three top-level categories. Unsafe Supervision and Operator Conditions are latent 
conditions while unsafe Acts of the Operator are active failures. Latent conditions are the 
result of decisions made by individuals who are not in direct control of the system 
(Zotov, 1996), and Active failures are those typically produced by “front-line” operators 
of the system (Reasons, 1998). 

Drawing from HFACS a maintenance extension (ME) taxonomy was developed 
to classify causal factors that contribute to Maintenance Related Mishaps (MRMs). This 
addition to HFACS consists of four broad human error categories (see Figure 2): 
Supervisory Conditions (latent), Working Conditions (latent), Maintainer Conditions 
(latent), and Maintainer Acts (active). Under the umbrella of organizational climate are 
the latent factors, Supervisory, Maintainer, and Working Conditions that can impact a 
maintainer’s performance and can contribute to an active failure, an unsafe maintainer 
act. These acts may directly lead to an incident (mishap or injury). The unsafe 
maintainer act may also become a latent maintenance condition, which the aircrew may 
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encounter while operating the aircraft. In addition, latent supervisory conditions may 
also lead directly to maintenance conditions. 

Examples of how this taxonomy is applied for the first and second order error 
types are now given. Latent Supervisory Conditions that can contribute to an active 
failure include both unforeseen conditions and those conditions caused by the squadron 
supervisory environment. An example of unforeseen supervisory condition is an engine 
that falls off of a stand due to an unforeseen hazard of high seas state. An example of a 
squadron supervisory condition is a supervisor who does not ensure that maintenance 
personnel are wearing required personal protective gear. 



Figure 2. HFACS Maintenace Extension Diagram 


Latent Maintainer Conditions that can contribute to an active failure include 
medical, crew coordination, and readiness. An example of a maintainer medical 
condition is a maintainer who has a marital problem and cannot focus on a maintenance 
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action. An example of maintainer crew coordination is a maintainer who performs a task, 
counter to standard procedures, because the maintainer is overly submissive to a superior. 
An example of maintainer readiness conditions is a maintainer who is working on an 
aircraft skipped the requisite on the job training evolution. 

Latent Working Conditions that contribute to an active failure include 
environmental, equipment and workspace. An example of environmental working 
conditions is a maintainer who is working at night on the flightline does not see a tool he 
left behind. And example of equipment working conditions is a maintainer who is using 
a defective test set does not precheck it before troubleshooting. An example of 
workspace working conditions is a maintainer who is working in a hangar bay cannot 
properly position the maintenance stand. 

Maintainer Acts are active failures, which directly or indirectly cause accidents, 
or lead to Latent Maintenance Condition, the include errors and violations. An example 
of errors in maintainer acts is a maintainer who is very familiar with a procedure might 
reverse steps in a sequence. An example of violations in maintainer acts is a maintainer 
who engages in practices, condoned by management, that bend the rules. 
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Table 1 shows the complete HFACS-ME categories having three orders: first, second 
and third that reflect a decomposition of the error types from macro to micro (Schmidt et 
al, 1998). 


First Order 

Second Order 

Third Order 

Supervisory 

Conditions 

Unforeseen 

Hazardous Operations 
Inadequate Documentation 
Inadequate Design 


Squadron 

Inadequate Supervision 
Inappropriate Operations 
Failed to Correct Problem 
Supervisory Violations 

Maintainer 

Conditions 

Medical 

Mental State 

Physical State 
Physical/Mental Limitation 


Crew Coordination 

Communication 

Assertiveness 

Adaptability/Flexibility 


Readiness 

Preparation/Training 

Qualification/Certification 

Violation 

Working 

Conditions 

Environment 

Lighting/Light 

Exposure/W eather 
Environmental Hazards 


Equipment 

Damaged 

Unavailable 

Dated/Uncertified 


Workspace 

Confining 

Obstructed 

Inaccessible 

Maintainer 

Acts 

Error 

Attention 

Memory 

Rule/Knowledge 


Violation 

Routine 

Infraction 

Exceptional 


Table 1. HFACS Maintenance Extension Categories 


Organizational climate can effect the three types of universal human activity that 
are common in hazardous environments: control under normal conditions, control under 
emergency conditions and maintenance-related activities. Reason (1997) defines 
maintenance-related to include unscheduled repairs, inspections, planned preventive 
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maintenance and calibration and testing. He further analyzes these activities and 
contends that it would be reasonable to expect that an activity scoring “high” in all three 
areas is the most likely to have a human performance problem. Of those universal human 
activities, Reason (1997) shows that these maintenance-related errors pose the largest 
human factors problem (see Table 2). 


Activity 

‘Hands on’ 

Criticality 

Frequency 

Normal control 

Low 

Moderate 

High 

Emergency control 

Moderate 

High 

Low 

Maintenance-related 

High 

High 

High 


Table 2. The Relative Likelihood of Human Performance 
Problems in the Universal Human Activities 


With this analysis complete Reason (1997) looks at which aspect of maintenance 
is most likely to be associated with “less-than-adequate” human performance. He 
concludes “that regardless of the domain, all maintenance-related activities require the 
removal of fastenings and the disassembly of components, followed by their reassembly 
and installation” the latter of which he identifies as the biggest problem (see Figure 3). 



Figure 3, The bolt-and-nuts example 


Reason (1999) argues that since there is only one way to remove the nuts, the chances of 
making an error are greatly reduced. However, in the case of installation there are over 
40,000 ways in which the nuts can be reassembled in the wrong order. While this 
example may seem simple, it is also supported by available evidence from leading 
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aircraft manufactures that report that installation errors comprise over 70 percent of the 
contributing factors to maintenance-related events. 

Reason (1997) further explains, errors committed by pilots, control room 
operators and other “sharp-end” personnel may be the last straw leading to an 
organizational mishap. However, it is the latent conditions created by maintenance- 
related mistakes that create the proper environment to start an accident in motion or make 
it impossible to recover when the error presents itself (Reason, 1997). By the same 
token, even though Reason (1997) paints a rather bleak picture with regard to 
maintenance-related errors, he states that they also offer the greatest opportunity for 
human factors improvements. 

To identify aviation safety concerns related to maintenance, Schmorrow (1998) 
examines Naval Aviator maintenance mishaps using HFACS-ME. His research, based 
on a prior study by Schmidt, Schmorrow and Hardee (1998), develops a method to 
quantitatively determine the existence of significant patterns of human error involved in 
flight mishaps. Using HFACS-ME, Teeters (1999) considers MRMs, Hazardous Reports 
(HAZREPS) and Personal Injury Reports (PIPs). He shows that if reductions in causal 
factors can be achieved, then the numbers of incidents related to these causal factors 
should be reduced. 

E. SUMMARY 

In the face of diminishing budgets and reduced manning, during the past decade 
Naval Aviation has been faced with the problem of maintaining operational readiness in a 
“make do” environment (Schmorrow, 1998). HROs such as Naval Aviation with their 
ever-increasing reliance on advanced and complex systems need ways to identify and 
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reduce human errors. The HFQMB with its initial focus on aircrew error has expanded 
the scope of effort to include maintenance-related errors. Two tools used in this effort, 
are the MCAS and HFACS-ME. Developed through coordinated efforts between Air 
Forces Pacific, Naval Safety Center, and the School of Aviation Safety at the Naval 
Postgraduate School, it has been shown that these tools are effected in 1) effectively 
measuring and organization’s safety climate and 2) classifying the errors present in those 
climates. It is hypothesized, that through the use of these devices one could assess a units 
risk and through analysis of the data recommend a preventive course of action to prevent 
a mishap from occurring Naval Aviation organizations. 
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III. METHODOLOGY 


A. RESEARCH APPROACH 


This study involves the analysis of data obtained from a 43 item Maintenance 
Climate Assessment Survey (MCAS), given to maintenance personal of the 3 rd MAW. 
This survey is based on a model of HROs. The survey results are analyzed to determine 
the surveys validity and reliability. In addition, the survey results are then used to 
determine if they can be associated with observed maintenance-related incidents (MRIs). 


MRIs are classified in accordance with the Human Factors Analysis and Classification 


System - Maintenance Extension (HFACS-ME) taxonomy to observe human error 


frequency and patterns. 

B. DATA COLLECTION 


1. Subjects 


A total of 976 officer and enlisted personnel involved in aviation maintenance in 
the 3 rd Marine Air Wing (MAW) participate in this study. The 3 rd MAW units are 
located at Marine Corps Air Station Miramar, California, Camp Pendleton, California, 
and Marine Corps Air Station Yuma, Arizona (See Figure 4 for the 3 rd MAW command 


structure). 




— MAG 11 (Miramar) 

-MALS 11 

-VMFA 121 

-VMFA 225 

-VMFA 242 

-VMFAT 101 

-VMFA 232 

-VMFA 314 

—VMFA 323 
-VMGR 352 


- MAG 13 (Yuma) 

—MALS 13 

-VMA211 

-VMA214 

—VMA311 
-VMA513 


— MAG 16 (Miramar) 

-MALS 16 

-HMH 465 

-HMH 462 

-HMH 361 

-HMM466 

-HMM 161 

—HMM 163 

-HMM 165 

-HMM 166 


— MAG 39 (Pendleton) 

—MALS 39 
—HMLA 169 
—HMLA 267 
—HMT 303 
—HMLA 367 
—HMLA 369 
—HMMT164 
—HMM 268 
-HMM 364 


Figure 4. 3rd MAW Command Structure 
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Four mishaps and five hazard reports (HAZREPs) data from the 3 rd MAW 
collected by the Naval Safety Center are used. The accidents in these reports occur 
between January 1998 to December 1999. 

2. Instruments 

a. MCAS 

The MCAS a self-administered, group survey consisting of six 
demographic and 43 maintenance related items. The maintenance items of the MCAS are 
grouped into six categories or components: (process auditing, reward system, quality, risk 
management, command and control, and communication/functional relationships). The 
survey uses a five point Likert rating scale with verbal anchors as follows: Strongly 
Disagree, Disagree, Neutral, Agree, and Strongly Agree. 

b. HFACS-ME 

HFACS-ME is used to classify casual factors that contribute to MRMs. 
This device uses a taxonomy that consists of four broad human error categories: 
Supervisory Conditions (latent), Working Conditions (latent), Maintainer Conditions 
(latent), and Maintainer Acts (active). This “first order effect” is further divided into 
second order and third order effects that decompose the error types from a macro to 
micro perspective. Since only first and second order casual factors Table 2 shows the 
reduced HFACS-ME categories. The mishap and HAZREP data is classified into first 
order causal factors that contribute to the MRMs using the Human Factors Accident 
Classification System - Maintenance Extension (HFACS-ME). 
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First Order 

Second Order 

Supervisory 

Conditions 

Unforeseen 

Squadron 

Maintainer 

Conditions 

Medical 

Crew Coordination 

Readiness 

Working 

Conditions 

Environment 

Equipment 

Workspace 

Maintainer 

Acts 

| Error 

Violation 


Table 3. HFACS ME 1 st & 2 nd Order Categories 
3. Procedure 
a. MCAS 

The survey is administered on site and in a group setting at the various 
participating squadrons of the 3 rd MAW. Additionally, the survey is given in conjunction 
with a scheduled maintenance safety presentation on human factors issues in aviation. 
The squadrons are in various stages of training and operational tasking at the time of the 
survey. Differences in operational tasking account for much of the variability in the 
number of surveys collected from each squadron. For example, a squadron not in the area 
due to deployment during the time of the survey would not have been included in this 
study. The potential MCAS respondents are briefed on the survey and its purpose. 
Questions that arise pertaining to the survey are answered by the survey administer. All 
surveys are collected immediately upon completion to allow for maximum 
accountability. 
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b. Mishap /HAZREP Data 

The Naval Safety Center (NAVSAFCEN) database was queried for all 


Marine Forces Pacific (MARFORPAC) incidents between Fiscal Year (FY) 1997 to 
February 2000. 

C. DATA ANALYSIS 

I. Data Tabulation 

a. MCAS 

Survey data sheets are scanned into a text file using a Scantron 8200 with 
Scanform version 3.21 software on an IMB compatible computer running Windows 3.1. 
The text file is converted into Microsoft Excel 2000 format. The spreadsheet consists of 
976 rows of respondents and 51 columns containing demographic variables and survey 
responses. The first eight columns contain the demographic variables and the remaining 
43 columns hold the subjects responses to questions 1 thru 43 of the survey. The 
demographic variables are: 1) aircraft community, 2) squadron number, 3) rank, 4) total 
years of aviation maintenance experience, 5) work center and 6) shift. All demographic 
information is then replaced by three variables indicating the squadron number, type 
aircraft and marine air group (MAG) from which the survey results are obtained. The 43 
columns are further partitioned into six sets or components. The six components are: 1) 
Process Auditing (questions 1-6), 2) Reward Systems and Safety Climate (questions 7- 
14), 3) Quality Assurance (questions 15-20), 4) Risk Management (questions 21-29), 5) 
Command and Control (questions 30-37), and 6) Communications (questions 38-43). 
The responses in each column are scored as 1,2,3,4 or 5 corresponding to the Likert scale 
Strongly Disagree (E), Disagree (D), Neutral (C), Agree (B), Strongly Agree (A), 
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respectively. Question 21 is negative in wording compared to all other questions of the 
survey. Survey questionnaire items that have no response are left blank. Questionnaires 
that have blank entries for any question or item, or for which the same answer is given for 
all questions are eliminated. A complete list of removed data is available in the results 
section. 

b. HFACS-ME 

A request is made to Naval Safety Center to query the database in order to 
provide all maintenance-related mishap and HAZREP data for all Marine Forces Pacific 
commands between the years 1996 to February 2000. This query generates a text file 
from which is culled all mishap and HAZREP data for commands of the 3 rd MAW from 
October 1997 to February 2000. These are then divided into Class A, B, C and HAZREP 
categories and are then evaluated into the HFACS-ME first and second order casual 
factors by the Naval Postgraduate School for Aviation Safety. Finally, simple counts are 
generated for the first and second order causal factors and these factors are then weighted 
to give an overall accident value for each squadron, type and MAG. These values are 
used to create a 3 rd MAW maintenance-related error table. 

2. Statistical Analysis 

a. MCAS 

Summary statistics are computed for each squadron, type aircraft and 
MAG in order to describe basic and general information about the question results. In 
particular, mean response for each question is computer by squadron, type aircraft and 
MAG. In addition, the questions within each of the six MOSE components are averaged 
for each respondent. These averages are then used as the independent variable in an 


25 






Analysis of Variance (ANOVA) to see if there is a difference in expected responses for 
different squadrons and for different MOSE components. It is noted that non-parametric 
analysis could also be done by replacing Likert scores with ranks before doing the 
ANOVA. However such non-parametric methods are a form of scoring and all methods 
are considered problematic (Kimeldorf, Sampson & Whitaker, 1992). Therefore, the 
very errors that a non-parametric approach might attempt to correct might in and of 
themselves introduce new unforeseen errors along with making the resulting analysis 
difficult to interpret to the population under study. Additionally, standard deviations are 
tabulated for the MOSE components and individual questions for each squadron, type 
aircraft and MAG. The S-Plus 2000 (Mathsoft, 1999) program is used for multivariate 
statistical analysis. Principal Component methodologies are used to validate the stability 
of the MCAS with a new population. 

b. HFACS-ME 

Summary statistics are developed for each squadron in order to describe 
basic and general information about the maintenance-related error. Simple linear 
regression is conducted with mean MCAS score as the independent variable and 
weighted MRI as the dependent variable. In addition, once complete an ad hoc procedure 
is conducted to see if the perceived safety climate predicted by the MCAS is related to an 
appropriate level of maintenance-related errors for those squadrons. 
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IV. RESULTS 


A. MCAS DESCRIPTIVE STATISTICS 

1. Sample 

Maintenance Climate Assessment Surveys (MCAS) are collected from 977 
military personnel serving in squadrons of the 3 rd Marine Air Wing (MAW). Of these, 
235 were in Marine Aviation Logistics Squadrons (MALS) and these surveys are not 
included in the study because these squadrons provide intermediate maintenance and do 
not have aircraft or aircrew directly assigned to them. Additionally, 61 surveys are 
removed from this remaining group either because of missing values (55) or the same 
response to all 43 questions (six). The remaining 681 surveys will be addressed in the 
MCAS Results section of this thesis. 

2. Analysis of Removed Surveys 

An effort is taken to determine if removing the 61 surveys with missing values or 
constant response introduces bias in the Principal Component Analysis. Of the 55 
surveys removed due to missing values, there are a total of 74 questions left blank. Table 
4 shows the questions associated with a missing value and the number of times that the 
particular question is not answered. Of note, are questions 17, 29, 36, 42 and 43, they 
are: 17) “QARs/CDIs sign-off after required actions are complete and are not pressured 
by supervisors to sign-off,” 29) “Based upon my command’s current assets/manning it is 
not over-committed,” 36) “All maintenance evolutions are properly briefed, supervised, 
and staffed by qualified personnel,” 42) “Maintenance Control troubleshoots/resolves 
gripes before flight,” and 43) “Maintainers are briefed on potential hazards associated 
with maintenance activities. These questions account for approximately 64 percent of the 
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missing values, with question 43 accounting for nearly 33 percent. In addition, 11 out of 
the 12 people who fail to answer question 42 also fail to answer question 43. According 
to Reasons (1998) this effect is observed for questions at the end of surveys producing 
what he calls “end effect.” Because questions 42, and 43 are the only questions on the 
last page of the survey this effect is amplified. 


# 

Questions 


3,5,9,10,11,14,15,16,18,20,24,26,32,33,37,40 

1 

1,2,4,6,8,12,13,19,21,22,27,28,30,31,34,35,39 

2 

7,23,25,38,41 

3 

17 

4 

29,36 

12 

42 

24 

43 


Table 4. Number Response Missing by Question 


To see if the 61 surveys differ from the other 681 surveys, the demographic variables 
of rank and squadron are compared. Table 5 lists the frequency and percentages of the 
surveys removed by rank. In Figure 5, these are compared to the surveys retained in the 
analysis. This figure shows that those surveys removed are representative of the 
remaining surveys used for the Principal Component Analysis and no bias should be 
introduced by their removal. 


Blank 

El-3 

E4-5 

E6-9 

Officer 

# 

% 

# 

% 

# 

% 

# 

% 

# 

% 

1 

1.7% 

32 

53.3% 

21 


6 

10.0% 

0 

0.0% 


Table 5. Percentage and Count of Removed Surveys by 
Participant Rank Response 
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Table 6 shows the frequency of surveys removed from a particular squadron. The 
tables shows a range of removed surveys between two percent for squadron B and 17 
percent for squadron F with an overall average of 10 percent surveys that have either a 
missing value or the same response for all questions. 



Figure 5. Comparison of Retained and Removed Surveys by Rank 


A 

B 

C 

E 

F 

G 

H 

I 

J 

K 

L 

M 


Q 





13% 

17% 

12% 

11% 

11% 

11% 

9% 

16% 

7% 





Table 6. Frequency by Squadrons of Surveys Removed 


3. Demographics 

The number of survey participants are given by rank and squadron in Table 7. In 
this table, participant squadrons and Marine Air Groups (MAGs) are randomly assigned 
letters in place of the squadron identifier. Approximately 84.9 percent of those 
participating in the MCAS are enlisted maintained serving in the ranks of El - E5. This 
percentage agrees with the 79.2% observed by Goodrum (1999) in his study of squadrons 
of the Fleet Logistics Support Wing and the 81% observed by Oneto (1999) in his study 
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of aircraft communities within the Naval Air Reserve Force. Similar proportions for all 
squadrons are observed for the MCAS participants except those units that contribute a 
low number of surveys as shown in Figure 6. Histograms of other descriptive 
demographic variables such as: 1) Total Years Maintenance Experience, 2) Work Center, 


and 3) Shift are given in Appendix (C). 



Blank 

e: 

1-3 

E4-5 

E6-9 

Officer 


Squadron 

# 

% 

# 

% 

# 

% 

# 

% 

# 

% 

Total 

A 

2 







14.5% 

0 


Kis 

B 





Si 

40.5% 

2 

4.8% 

1 



C 

1 

MM 


43.5% 

34 

keui 


4.3% 

1 

1.4% 

r 69 

D 



9 


mm 

lfcWE10« 

1 

4.8% 

0 


mm 

E 







3 

37.5% 

mm 

0.0% 

8 

F 

4 

6.2% 


60.0% 

20 

30.8% 

2 

3.1% 

■9 

0.0% 

65 

G 

1 

2.0% 

16 

32.0% 

22 

EEKflgg 

8 

16.0% 

3 

6.0% 

50 

H 

0 

0.0% 


47.4% 

5 


3 

15.8% 

2 


mm 

I 




48.6% 



9 

12.5% 

0 

0.0% 

72 

J 

0 


4 

44.4% 

4 


1 

11.1% 

0 

0.0% 

9 

K 

0 



81.8% 

2 



0.0% 

0 

0.0% 

11 

L 




37.8% 

19 

BEE 

3 

8.1% 

1 



M 

3 

3.4% 

44 

50.0% 

30 


9 

10.2% 

2 



N 




0.0% 

1 


7 

77.8% 

1 

11.1% 


0 

3 

5.6% 

15 

27.8% 

29 

53.7% 

6 

11.1% 

1 

mm 


P 

mm 



38.6% 

29 

50.9% 

5 

8.8% 

0 

0.0% 

57 

Q 

0 

0.0% 





5 

62.5% 

0 

0.0% 

8 

Total 

15 

2.2% 

297 

43.6% 

281 

41.3% 

76 

11.2% 

12 

1.8% 

681 


Table 7. Number and Percentages of Respondents by Rank for each Squadron 



Figure 6. Percentage of MCAS Respondents’ with Rank E1-E5 for each Squadron 
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4. MCAS MOSE Component Statistics 

Table 8 displays the six MOSE categories and respective MCAS questions that 
comprise each component. Table 9 displays the mean response by component and 
squadrons of the 3 rd MAW. The 3 rd MAW question averages by squadron and the MOSE 
component statistics for the MAGS and type aircraft can be found in Appendices C and D 
respectively. 


MOSE Category 

Questions 

Process Auditing (PA) 

1,2,3,4, 5,6 

Reward System & Safety Climate (RS) 

7,8,9,10,11, 12,13,14 

Quality Assurance (QA) 

15,16,17,18,19,20 

Risk Management (RM) 

21,22,23,24, 25,26,27,28,29 

Command & Control (CC) 

30,31,32, 33, 34, 35,36,37 

Communication / Functional Relationships 

38,39,40,41,42,43 


Table 8. MOSE Categories and Questions 



Table 9. Mean MCAS Response by Component and Squadron of 3 rd MAW 
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Variances 


B. MCAS PRINCIPAL COMPONENT ANALYSIS 

Principal Component Analysis (Hamilton, 1992) is performed on the survey data 
to test whether the factors that make up the survey account equally for most of the 
variability of the data. Using S-Plus 4.5, the command “princomp (MAW, cor = F)” is 
used to produce the results of the principal component analysis. The Principal 
Component is defined as the 681 by 43 matrix of survey responses (one row per survey 
and one column per question). In addition, the analysis is based on the covariance matrix 
rather than the correlation matrix. The covariance matrix is used since all questions are 
on a common scale. Principal Component Analysis is often used to reduce the number of 
explanatory variables (questions), however it is being used here to revalidate the survey 
by showing no reduction is required. A scree-plot is used to display the output of the 
principal component analysis (see Figure 7). 



Figure 7. Principal Component Scree-Plot. 
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In this figure, the variance contributed by each component is plotted and the 
cumulative variance given. Only the first 10 of 43 components of the Principal 
Component Analysis are displayed. The principal component loadings are the 
coefficients of the principal components transformations. Observe that the first 
component, which is the maximum variance of the data, explains 32.7 percent of the total 
variance of the survey. As shown in Appendix E all questions load equally across the 
first component. Additionally, with a large difference between the 1 st and 2 nd component 
and a gradual decrease thereafter, this suggests that the survey is dominated by the first 
component and that linear combinations of the questions combine for the remaining 
variance. This analysis has one important implication, the questions in the survey are 
contributing equally to the overall assessment of maintenance climate, and the individual 
questions and their interactions are required to capture this climate. 

C. MCAS MOSE COMPONENT ANALYSIS 

Only squadrons with at least 25 surveys are used to do further explanatory 
analysis, Analysis Of Variance (ANOVA) and Multiple Comparison Analysis 
(MANOVA) on the MOSE components. Since most squadrons have a maintainer census 
between 90 and 150, with an average of 120, it is deemed necessary to capture at least 20 
percent of the maintainers’ perceptions in order to draw any meaningful conclusions from 
the survey. The data set is reduced by 85 surveys to 596 by eliminating squadrons D, E, 
H, J, K, N and Q. This reduction eliminates the ability to compare differences between 
MAGs and type aircraft. Table 10 shows the remaining squadrons and their MOSE 
component statistics. 
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Sqdrn 

Process 

Auditing 

Reward 

System 

Quality 

Assurance 

Risk 

Management 

Command 
& Control 

Comms/ 
Func Rel 

Sqdrn 

Mean 

A 

3.75 

3.41 

3.63 

3.12 

3.50 

3.43 

3.47 

B 

4.07 

3.53 

3.94 

3.47 

3.85 

3.66 

3.75 

C 

4.00 

3.34 

3.56 

3.30 

3.40 

3.37 

3.50 

F 

4.02 

3.67 

3.60 

3.65 

3.68 

3.55 

3.70 

G 

3.79 

3.37 

3.42 

2.98 

3.33 

3.21 

3.35 

I 

3.98 

3.39 

3.52 

3.28 

3.51 

3.53 

3.54 

L 

4.02 

3.74 

3.99 

3.56 

3.83 

3.92 

3.84 

M 

3.61 

3.25 

3.37 

2.99 

3.17 

3.19 

3.26 

O 

3.58 

3.21 

3.32 

2.82 

3.18 

3.20 

3.22 

P 

3.74 

3.28 

3.66 

3.20 

3.45 

3.63 

3.49 

Component 

Mean 

3.86 

3.42 

3.60 

3.24 

3.49 

3.47 

3.51 


Table 10. Mean Survey Response by Component and Squadron 
for Squadrons with at least 25 Surveys 

Figures 8 and 9 are bar charts showing the MOSE mean responses of the 
components and MCAS mean responses for the squadrons, respectively. The average 
Likert scale response (1 = “Strongly Disagree, 2 = “Disagree”, 3 = “Neutral”, 4. “Agree,” 
5 = “Strongly Agree”) for both the MOSE components and the squadrons was 3.51. Of 
note is the mean for Risk Management (RM) is 3.24 with a low of 2.82 (O) and a high of 
3.65 (F). The highest scored component is Process Auditing (PA) with a mean of 3.86 
and a high of 4.07 (B) and low of 3.58 (O). For the squadrons notice that G, M and O are 
well below the mean while the high for all is 3.84 (L) and the low is 3.22 (O). 



Figure 8. MOSE Component Means with Average 
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3.84 


3.9 n 



ABCFG I LMOP AVG 
Figure 9. Squadron MCAS Means with Average 


1. Analysis of Variance 

A two-factor ANOVA is conducted to determine if either the squadron or the 
MOSE component show any effect on the mean survey response. Let 7 represent the 
levels of the first factor (squadrons) and 7 represent the levels of the second factor 
(MOSE components) then in this data set 7=10 (the ten squadrons) and 7=6 (the six 
MOSE components), and there are 7*7=60 possible cross classifications that make up the 
levels of the two factors. The dependent variable is the mean question response given in 
Table 10. 

Appendix F has a detailed listing of the S-PLUS code and associated output in the 
ANOVA analysis on the data in Table 10. The results of the analysis on the two factors 
with average as the response is displayed on the next page in Table 11. 
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df 

Sum of Sq 

Mean Sq 

F 

Pf-value 

Squad 

9 

2.27 

0.252 

26.7 

3.22 x 10' 15 

Comp 

5 

2.13 


45.1 

2.20 x 10’ 15 

Residuals 

45 

0.43 





Table 11. Analysis of Variance on Squadron and MOSE Factors 


The p-values for the null hypothesis that there is no squadron effect and there is 
no component effect are less than 10' 14 . Thus there is strong evidence that at least one 
squadron has a statistically different mean score, and there is strong evidence that at least 
one of the MOSE components do not yield the same mean responses. 

2. Multiple ANOVA Comparisons 

A multiple comparison test using Tukey’s Procedure (or “T” method) is 
conducted to determine which squadrons are different from one another. Utilizing the 
Studentized Range probability distribution, simultaneous confidence intervals for all 
pairwise comparisons were computed at a selected level of significance. In this case, the 
pairwise comparing of the means of all six MOSE components results in fifteen 
comparisons. The resulting confidence intervals are intervals for the values of all pairs of 
differences between true treatment means. Each interval that doesn’t include the value of 
zero yields the conclusion that the treatment means differ significantly (Devore, 1995). 

Appendix F has a detailed listing of the S-PLUS code and associated output for 
Tukey’s procedure. Figure 10 is the boxplot of mean responses for the 10 squadrons. 
Having the squadron means of 3.22 (0), 3.26 (M), 3.35 (G), 3.47 (A), 3.49 (P), 3.49 (C), 
3.54, (I), 3.70 (F), 3.75 (B), and 3.84 (L) arranged in increasing order, the graphical 
display of the confidence intervals (Figure 11) is used to obtain those pairings that 
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contain a value of zero. The results of this comparison is summarized graphically in 
Figure 12. Any squadrons not underscored by the same line, are statistically different 
from each other at a five percent level of significance. Figure 12 shows that O and M are 
different from all other squadrons except G. Squadrons A, P and C are different from O, 
M and F, B, and L. Finally, squadrons B and L are different from all other squadrons 
except F. Since G and F form the bridge between the three groups they are considered 
boundary squadrons having characteristics of two groups. 



Figure 10. Boxplots of Squadrons 
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response variable: Squad 


Figure 11. Tukey’s Pairwise Comparison of Squadrons 
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Figure 12. Identifying Statistically Different Squadrons 


Finally, a multiple comparison analysis is conducted on the MOSE component 
factor in the same manner as above. Again, having the MOSE component means of 3.24 
(RM), 3.42 (RS), 3.47 (CF), 3.49 (CC), 3.60 (QA), and 3.86 (PA) arranged in increasing 
order, the graphical display of the confidence intervals (Figure 13) is used to obtain those 
pairing that contain a value of zero. Those squadrons are then underlined as shown in 
Figure 14. Figure 14 shows that RM and PA are found to be different from all other 
components. QA is found to be different from all components except CC. 



response variable: Squad 

Figure 13. Tukey’s Pairwise Comparison of MOSE Components 
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RM RS CF CC QA PA 


Figure 14. Identifying Statistically Different MOSE Components 


D. HFACS-ME DESCRIPTIVE STATISTICS 

Between calendar years 1998 - 1999 the 3rd MAW experienced two class A, 
three class B, four class C Maintenance Related Incidents (MRIs) 13 HAZREPS. Table 
12 lists the MRMs with the associated cost of the mishap or HAZREP. During this time 
frame, squadrons D, L, M, P and Q reported no MRMs or HAZREPS. 



Class A 

Class B 

Class C 

HAZREP 

D 




$10,000 

B 





C 

$11,783,499 




E 


$227,350 

$360,515 

$195,730 

$10,000 
$1 0.000 

a 

$24,397,330 



$10,000 

$10,000 

o 




$10,000 

H 




$10,000 

I 




$10,000 

$10,000 

J 




$10,000 

K 



$116,751 

$77,005 

$10,000 

$10,000 

N 




$10,000 

0 


$120,487 




Table 12. Class A, B and C MRMs and HAZREPS for the 3 rd MAW 
between Jan 1998 - Dec 1999 


Using HFACS-ME to identify the casual factors in the MRIs and HAZREPS the 
total first and second order casual factors for each squadron reporting an incident are 
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counted and are presented in Table 13. If there is no second order casual factor observed 
that column is removed. 



MA 

MC 

sc 

WC 


Error 

Violation 

Crew Coord 

Unforeseen 

Sqdrn 

Equipment 

A 

i 




i 


B 

i 

i 

i 


2 


C 

2 




1 


D 







E 

5 

i 


4 

1 

i 

F 

2 

2 


1 



G 

1 






I 

1 

1 


1 

2 


J 

1 





i 

K 

4 

1 



3 


N 

1 






O 

1 

1 


3 

1 



Table 13. HFACS-ME 1 st and 2 nd Order Casual Factors for MRIs and 
HAZREPS of the 3 rd MAW between Jan 1998 - Dec 1999 


E. HFACS-ME / MCAS CORRELATION ANALYSIS 

Finally, an effort is undertaken to identify relationships between the average 
MCAS score for a squadron and their MRIs and HAZREPS. This is accomplished by 
first combining all MRIs and HAZREPS for each squadron, and then dividing by the total 
flight hours for the past two years, giving an incident rate per flight hour. The summary 
of this computation along with mean MCAS response for each squadron is given in Table 
14. A simple linear regression of MCAS mean response versus incident rate from Table 
14 is performed. Note that, only those squadrons who completed 25 or more surveys are 
included. Figure 15 plots the linear regression with mean MCAS as the predictor 
variable and incident rate per flight hour as the response variable. Squadron F can be 
observed as an influential point greatly affecting the slope coefficient in the regression. 
Figure 16 gives the same linear regression with squadron F removed. Appendix G 
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contains the detailed results of both regressions. For the model containing F the 
estimated slope coefficient is 32.8 with an associated t-value of .672. For the model 
without F the estimated slope coefficient is -8.71 with an associated t value of -.395; 
neither gives slopes that are statistically different from zero. Thus, the results are 
inconclusive. There is not enough evidence to say that there is a relationship (either 
positive or negative) between mean MCAS response and incident rate. Similar results are 
obtained when incidents are weighted according to cost. 



# Incidents 

Flight Hours (FH) 

Incident / FH 

Mean MCAS 

A 

i 

7257.2 

13.8 

3.47 

B 

1 

12076.6 

8.3 

3.75 

C 

i 

6403.9 

15.6 

3.49 

F 

3 

3128.9 

95.9 

3.69 

G 

1 

6209.3 

16.1 

3.35 

I 

2 

5246.5 

38.1 

3.53 

L 

0 

2378.2 

0.0 

3.84 

M 

0 

3052.9 

0.0 

3.26 

O 

1 

6402 

15.6 

3.22 

P 

0 

5783.2 

0.0 

3.49 


Table 14.3 rd MAW Incident, Flight Hour, Incident Rate and 
Mean MCAS data from Jan 1998 - Dec 1999 
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Incident Rate/FH Incident Rate/FH 



Figure 15. Simple Linear Regression with Mean MCAS as 
Predictor and Incident Rate/FH as Response 



3.10 3.20 3.30 3.40 3.50 3.60 3.70 3.80 3.90 

Mean MCAS 

Figure 16. Simple Linear Regression with Mean MCAS as 
Predictor and Incident Rate/FH as Response minus F 
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Table 15 summarizes an ad hoc analysis done by the School of Aviation Safety at 
the Naval Postgraduate School on the 10 MRIs of the 3 rd MAW and the MCAS MOSE 
components. In this analysis, two people with expertise in general aviation and aviation 
safety review 10 MRIs. In Table 15, an “X” in the cell indicates that at least one question 
in a particular component addresses the casual factors associated with the incident and 
that both persons evaluating the MRI at the School of Aviation Safety at the Naval 
Postgraduate School agree that the incident has that MOSE component associated with at 
least one of the causal factors in the incident. For example, the MOSE component, PA, 
has at least one question that addresses a casual factor in incidents “1”, “2”, “4”, and “6” 
and incident “1” has a PA and CF MOSE component in the causal factors. 

This table indicates that based on expert judgement, there appears to be a 
relationship between the MOSE components of the MCAS and the MRIs. Every incident 
has at least one causal factor associated with a MOSE component. Of note is the high 
occurrence of the MOSE component QA showing up in eight out of ten incidents. 
Likewise, the MOSE component RS has a low occurrence showing up in only one out of 
ten incidents. 




INCIDENTS I 

MOSE 

COMPONENTS 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

PA 

X 

X 


X 


X 





RS 



X 








QA 


X 

X 

X 

X 

X 

X 


X 

X 

RM 





X 





X 

CC 



X 

X 




X 


X 

CF 

X 



X 








Table 15.3 rd MAW Incident Factors Captured by MCAS MOSE Components 
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V. SUMMARY, CONCLUSION, AND RECOMMENDATIONS 


A. SUMMARY 

Squadron commanders have a fundamental responsibility to safeguard personnel 
and material resources while at the same time minimizing risk while accomplishing 
assigned missions. Dramatic strides in reducing the Class A Flight Mishap (FM) rate 
have occurred and only recently have these reductions begun to level off forcing Naval 
Aviation to look at alternate ways to continue the reduction. Recognizing that one in five 
Naval Aviation Class A FMs has maintenance-related casual factors a recent effort has 
begun to address aircraft maintenance issues in order to continue the Class A FM 
reductions. Additionally major and minor mishaps have come under review and for this 
study were called maintenance-related incidences (MRIs). 

The primary focus of this study was to analyze the Maintenance Climate 
Assessment Survey (MCAS) and its ability to capture maintainer perceptions on 3 rd 
Marine Air Wing (MAW) maintenance operations. Of secondary interest, was the 
application of the Human Factors Accident Classification System - Maintenance 
Extension (HFACS-ME) to the 3 rd MAW to explore their human errors and to assess if 
there is a relationship between their MRIs and the MCAS. Both of these vehicles were 
looked at in an effort to: 1) obtain 3 rd MAW maintainer perception of maintenance 
operations safety, 2) evaluate the ability of the MCAS to detect differences in squadron 
maintenance safety climate and how it is associated with the MRI histories and 3) 
determine if human error is a significant factor in 3 rd MAW MRIs using HFACS-ME. 
This research involved the collection of 977 survey results during the first part of Fiscal 
Year (FY) 2000 and the analysis of 21 MRIs from January 1998 to December 1999. 
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B. CONCLUSIONS 


The results of this study show that the MCAS can be utilized by Naval Aviation 
as a tool for effectively capturing a maintainer’s perceptions of safety in maintenance 
operations. Principal Component Analysis fails to identify any single MOSE component 
or question that was responsible for controlling the outcome of the survey. Through 
analysis of the removed surveys with missing values or the same response to all 43 
questions it was determined that question order effects were present. One minor finding 
was Question 21 was found to not load well in the first principal component due to the 
negative wording in the question. 

Additionally, through Analysis of Variance (ANOVA) and multiple comparison 
testing, it is determined that the MCAS is able to detect differences between both the 
squadron’s mean MCAS and MOSE component response. Through this analysis, it was 
discovered that Risk Management is a concern within the 3 rd MAW. It also highlighted a 
positive outlook with respect to the command Process Auditing. Unfortunately, this 
study was unable to conduct the same analysis at the Marine Air Group or Type Aircraft 
level due to an insufficient number of surveys across these levels of the 3 rd MAW. 

Finally, the analysis done on the Maintenance Related Incidences (MRIs) of the 
3 rd MAW using HFACS-ME was also limited by the number of surveys as well as a 
general low rate of MRIs. The fact that the safety climate of a squadron lasts about two 
years makes it impossible to conduct an in-depth analysis of the ordered causal factors 
relating to the MRIs in a one time study. The low t-value associated with the linear 
regression fit with squadron F included and removed lead one to conclude that this one 
study is insufficient to demonstrate a relationship between the two. However, based on 
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the content analysis of the MRIs and the MCAS MOSE components it is clear that the 
survey is addressing issues related to incidents of the 3 rd MAW. Of note is the low rate 
of the Reward System and Safety Climate (RS) component in the incidents of the 3 rd 
MAW but this could simply be due to chance with such a relatively low number of 
incidents. 

C. RECOMMENDATIONS 

Based on conclusions from the study, a number of recommendations concerning 
question order, minor MCAS modifications and administration and analysis of the survey 
are listed below: 

1. The 3 rd MAW should review their Risk Management maintenance processes. 
In addition this study can be used as a starting point for year over year 
comparison of 3 rd MAW MCAS and MRI data to determine to what degree 
there is a relationship between perceptions of maintenance safety and actual 
safety records 

2. Minor modifications of the MCAS need to be made including, 1) Positively 
wording question 21 2) randomizing the questions with respect to the MOSE 
components to reduce response order effects and 3) revisiting the questions of 
the Reward System component to ensure they are adequately addressing 
casual factors in MRIs. 

3. Since often times the survey was given under different circumstances, a more 
systematic administration of the MCAS needs to be developed. One method 
currently being developed by the School of Aviation Safety at the Naval 
Postgraduate School is on-line version of the MCAS similar to the Command 
Safety Assessment (CSA) currently being given to aircrew. This method has 
the advantage of real time analysis of surveys from a command. 

Implementation of these recommendations would lead to a more effective and useful 

MCAS for use in the future by other aviation commands. In addition, as previous 

research has shown intervention in the Risk Management practices of the 3 rd MAW could 

lead to substantial reductions in the MRIs associated with this casual factor. 
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While the above recommendations will positively affect the MCAS, further 
research is needed in the following areas: 

1. In order to reduce the response error that may be introduced by a persons 
background it is recommend that a study be undertaken to asses the 
relationship, if any, between a persons demographic and MCAS responses. 
To completely validate this survey, it needs to be shown that the MCAS 
responses are independent of the demographics of the respondent. 

2. Finally, to accurately assess the link between the MCAS and MRI rate a 
systematic study of squadrons and their mishap rate needs to be conducted. 

The benefits to this additional research, will be to make the MCAS a more effective tool 

in capturing maintainer perceptions on maintenance operations while continuing to strive 

for the ultimate safety goal of intervention by aviation commanders in real time to correct 

safety deficiencies that will eventually lead to a future incident. 
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APPENDIX A. 43-ITEM MAINTENACE CLIMATE ASSESSMENT SURVEY 

MAINTENANCE CLIMATE ASSESSMENT SURVEY fMCASt 

Purpose: The MCAS was designed to capture maintainer perceptions of maintenance operations as they 
relate to safety. Your responses help guide Naval Aviation’s on-going efforts to reduce aviation related 
mishaps. Thank vou in advance for vour participation ! 

Directions: Do not write on this form. Fill in all of your responses using the computer sheet provided. 
Fill in each box that corresponds to your response completely using a pencil. This is not a timed event, so 
answer each question carefully and honestly. Individual responses will not be reported, only compiled 
results will be provided to each squadron. 

Parti- Demographics has six items requesting unit and biographical data. This information will aid in the 
response analysis. NO attempts will be made to identify individuals. 

Partll- Perceptions has 43 questions pertaining to the maintenance operations. Please chose the response 
to each item that most correctly reflects your honest opinion. Responses are: 

A-Strongly Agree B-Agree C-Neutral D-Disagree E-Strongly Disagree 
Part I- Demographics 

Line 1 Fill in the numbered circle corresponding to your community? 

VMGR (1) VMA (2) VMFA (3) HMT (4) HMM(5) 
VMAQ (6) HMH (7) VMH (9) Other (9) () 

Line 2-4 Fill in the circles corresponding to your squadron number 

Line 5 Fill in the numbered circle corresponding with you rank 

(1) El-3 (2)E4-5 (3)E6-7 (4)E8-9 (5)W01-4 (6)01-03 (7)04-5 

Line 6 Fill in the numbered circle corresponding to your total years 

of Aviation Maintenance experience 
(1) <1 (2) 1-2 (3)3-5 (4)6-10 (5)11-15 (6)15-20 (7)20+ 

Line 7 Fill in the numbered circle corresponding to your work center 

(l)Powerplants (2)Airffames (3)Survival (4) Quality Assurance 
(5)Ordnance (6)Avionics (7)MAINT Control (8)Line (9)Other 

Line 8 Fill in the numbered circle corresponding to your shift 

(l)Day (2)Night 

Part II Perceptions 

Fill in the lettered circle that corresponds with your response to each item. 



SA 

A 

N 

D 

SD 

1 . The command adequately reviews and updates safety. 

(A) 

(B) 

(C) 

(D) 

(E) 


SA 

A 

N 

D 

SD 

2. The command monitors maintainer qualifications and has 

(A) 

(B) 

(C) 

(D) 

(E) 

a program that targets training deficiencies. 


SA 

A 

N 

D 

SD 

3. The command uses safety and medical staff to identify/ 



(A) 

(B) 

(C) 

(D) 

(E) 

manage personnel at risk. 


SA 

A 

N 

D 

SD 

4. CDIs/QARs routinely monitor maintenance evolutions. 

(A) 

(B) 

(C) 

(D) 

(E) 
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SA 

A 

N 

D 

SD 

5. Tool control is taken seriously in the command and 

(A) 

(B) 

(C) 

(D) 

(E) 

support equipment licensing is closely monitored. 


SA 

A 

N 

D 

SD 

6. Signing off personnel qualifications are taken seriously. 

(A) 

(B) 

(C) 

(D) 

(E) 


SA 

A 

N 

D 

SD 

7. Our command climate promotes safe maintenance. 

(A) 

(B) 

(C) 

(D) 

(E) 


SA 

A 

N 

D 

SD 

8. Supervisors discourage SOP, NAMP, or other procedure 

(A) 

(B) 

(C) 

(D) 

(E) 

violations and encourage reporting safety concerns. 


SA 

A 

N 

D 

SD 

9. Peer influence discourages SOP, NAMP, or other 

(A) 

(B) 

fC) 

(D) 


violations and individuals feel free to report them. 


SA 

A 

N 

D 

SD 

10. Violations of SOP, NAMP, or other procedures are not 

(A) 

(B) 

(C) 

(D) 

(E) 

common in this command. 


SA 

A 

N 

D 

SD 

11. The command recognizes individual safety achievement 

(A) 

(B) 

(C) 

(D) 

(E) 

through rewards and incentives. 


SA 

A 

N 

D 

SD 

12. Personnel are comfortable approaching supervisors 

(A) 

(B) 

(C) 

(D) 

(E) 

about personal problems/illness 


SA 

A 

N 

D 

SD 

13. Safety NCO, QAR, and CDI, are sought after billets. 

(A) 

(B) 

(C) 

(D) 

(E) 


SA 

A 

N 

D 

SD 

14. Unprofessional behavior is not tolerated in the command 

(A) 

(B) 

(C) 

(D) 

(E) 


SA 

A 

N 

D 

SD 

15. The command has a reputation for quality maintenance 

(A) 

(B) 

(C) 

(D) 

(E) 

and sets standards to maintain quality control. 


SA 

A 

N 

D 

SD 

16. QA and Safety are well respected, and are seen as 

(A) 

(B) 

(C) 

(D) 

(E) 

essential to mission accomplishment. 


SA 

A 

N 

D 

SD 

17. QARs/CDIs sign-off after required actions are complete 

(A) 

(B) 

(C) 

(D) 

(E) 

and are not pressured by supervisors to sign-off. 


SA 

A 

N 

D 

SD 

18. Maintenance on detachments is of the same quality 

(A) 

(B) 

(C) 

(D) 

(E) 

as that at home station. 


SA 

A 

N 

D 

SD 






19. Required publications/tools/equipment are available, 

(A) 

(B) 

(C) 

(D) 

(E) 

current/serviceable, and used. 


SA 

A 

N 

D 

SD 

20. QARs are helpful, and QA is not “feared” in my unit. 

(A) 

(B) 

(C) 

(D) 

(E) 


SA 

A 

N 

D 

SD 

21. Multiple job assignments and collateral duties adversely 

(A) 

(B) 

(C) 

(D) 

(E) 

affect maintenance. 


SA 

A 

N 

D 

SD 

22. Safety is part of maintenance planning, and additional 

(A) 

(B) 

(C) 

(D) 

(E) 

training/support is provided as needed. 
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23. Supervisors recognize unsafe conditions and manage 
hazards associated with maintenance and the flight-line. 

24. I am provided adequate resources, time, personnel to 
accomplish my job. 

25. Personnel turnover does not negatively impact the 
command’s ability to operate safely. 

26. Supervisors are more concerned with safe maintenance 
than the flight schedule, and do no permit cutting comers. 

27. Day/Night Check have equal workloads, and staffing is 
sufficient on each shift. 

28. Supervisors shield personnel from outside pressures and 
are aware of individual workload. 

29. Based upon my command’s current assets/manning it 
is not over-committed. 

30. My command temporarily restricts maintainers who are 
having a problems. 

31. Safety decisions are made at the proper levels and 
work center supervisor decisions are respected. 

32. Supervisors communicate command safety goals and 
are actively engaged in the safety program. 

33. Supervisors set the example for following to 
maintenance standards and ensure compliance. 

34. In my command safety is a key part of all maintenance 
operations, and all are responsible/accountable for safety. 

35. Safety education and training are comprehensive and 
effective. 

36. All maintenance evolutions are properly briefed, 
supervised, and staffed by qualified personnel. 

37. Maintenance Control is effective in managing all 
maintenance activities. 

38. Good communication exists up/down the chain of 
command. 

39. I get all the information I need to do my job safely. 


40. Work center supervisors coordinate their actions. 


41. My command has effective pass-down between shifts. 
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SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N . 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 



42. Maintenance Control troubleshoots/resolves gripes before 
flight. 

43. Maintained are briefed on potential hazards associated 
with maintenance activities. 


SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 

SA 

A 

N 

D 

SD 

(A) 

(B) 

(C) 

(D) 

(E) 
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APPENDIX B. MODEL OF SAFETY EFFECTIVENESS COMPONENTS 


COMPONENT 1: PROCESS AUDITING 

1. The command adequately reviews and updates safety practices. 

2. The command monitors maintainer qualifications and has a program that targets 
training deficiencies. 

3. The command uses safety and medical staff to identify/manage personnel at risk. 

4. CDIs/QARs routinely monitor maintenance evolutions. 

5. Tool Control is taken seriously in the command and support equipment licensing is 
closely monitored. 

6. Signing personal qualifications are taken seriously. 

COMPONENT 2: REWARD SYSTEM and SAFETY CLIMATE 

1. Our command climate promotes safe maintenance and flight operations. 

2. Supervisors discourage SOP, NAMP or other procedure violations and encourage 
reporting safety concern. 

3. Peer influence discourages SOP, NAMP or other violations and individuals feel free 
to report them. 

4. Violations of SOP, NAMP or other procedures are not common in this command. 

5. The command recognizes individual safety achievement through rewards and 
incentives. 

6. Personnel are comfortable approaching supervisors about personal problems/illness. 

7. Safety NCO, QAR, and CDI, are sought after billets. 

8. Unprofessional behavior is not tolerated in the command. 

COMPONENT 3: QUALITY ASSURANCE 

1. The command has a reputation for quality maintenance and has set standards to 
maintain quality control. 

2. QA and Safety are well respected, and are seen as essential to mi ssion 
accomplishment. 

3. Q ARs/CDIs sign-off after required actions are complete and are not pressured by 
supervisors to sign-off. 

4. Maintenance on detachments is the same quality as that at home station. 

5. Required publications/tools/equipment are available, current/serviceable, and used. 

6. QARs are helpful, and QA is not “feared” in my unit. 
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COMPONENT 4: RISK MANAGEMENT 


1. Multiple job assignments and collateral duties adversely affect maintenance. 

2. Safety is part of maintenance planning, and additional training/support is provided as 
needed. 

3. Supervisors recognize unsafe conditions and manage hazards associated with 
maintenance and the flight line. 

4. I am provided adequate resources, time, personnel to accomplish my job. 

5. Personnel turnover does not negatively impact the command’s ability to operate 
safely. 

6. Supervisors are more concerned with safe maintenance than the flight schedule, and 
do not permit cutting comers. 

7. Day/Night Check have equal workloads, and staffing is sufficient on each shift. 

8. Supervisors shield personnel from outside pressures and are aware of individual 
workload. 

9. Based upon my command’s current assets/manning it is not over-committed. 

COMPONENT 5: COMMAND AND CONTROL 

1. My command temporarily restricts maintained who are having problems. 

2. Safety decisions are made at the proper levels, work center supervisors decisions are 
respected. 

3. Supervisors communicate command safety goals and are actively engaged in the 
safety program. 

4. Supervisors set the example for following to maintenance standards and ensure 
compliance. 

5. In my command safety is a key part of all maintenance operations and all are 
responsible/accountable for safety. 

6. Safety education and training are comprehensive and effective. 

7. All maintenance evolutions are properly briefed, supervised, and staffed by qualified 
personnel. 

8. Maintenance control is effective in managing all maintenance activities. 

COMPONENT 6: COMMUNICATION / FUNCTIONAL RELATIONSHIPS 

1. Good communication exists up/down the chain of command. 

2. I get all the information I need to do my job safely. 

3. Work center supervisors coordinate their actions. 

4. My command has effective pass-down between shifts. 

5. Maintenance Control troubleshoots/resolves gripes before flight. 

6. Maintainers are briefed on potential hazards associated with maintenance activities. 



APPENDIX C. 3RD MAW QUESTION AVERAGES BY SQUADRON 


Sqdrn 

Ql 

Q2 

Q3 

Q4 

Q5 

Q6 

Q7 

Q8 

Q9 

[Q10 

Qll 

Q12 

Q13 

Q14 

Q15 

A 

3.76 

3.57 

3.43 

4.05 

3.81 

3.86 

3.86 

3.38 

3.29 


2.90 

3.81 

3.33 

3.43 

3.76 

B 


4.10 

3.38 

4.33 



3.95 




3.05 

3.81 

3.19 

3.57 

4.24 

C 


3.86 

3.19 

4.38 

4.33 

4.10 

4.00 

3.43 

3.05 

3.43 

2.48 

3.33 

3.48 

3.52 

3.76 

D 

3.90 

3.48 

3.24 


3.90 

3.62 



BE£1 


3.43 

3.71 

3.38 

3.00 

3.90 

Kflj 

BB3 




3.50 

3.88 





3.13 

4.00 

3.38 

4.25 

4.50 

F 

EB3 

3.90 

3.76 

Eld 

4.14 

3.86 

3.90 




3.29 

4.00 

3.76 

3.38 

3.81 

G 

4.16 

3.76 

3.14 

4.10 

3.81 

3.76 








3.62 

4.05 

H 

4.42 

3.89 

3.11 

4.26 

3.89 

3.74 

4.05 

3.79 

3.37 

3.68 




3.68 

4.53 

I 

4.14 




4.24 

4.05 

4.05 



3.52 




3.57 

4.24 

J 

4.22 

3.78 

3.11 

ESI 

4.22 

4.33 

4.22 

4.11 

3.56 





4.00 

4.44 

K 

3.73 

3.73 

3.18 

BEB 

4.09 

3.00 




3.00 

3.18 

3.00 

3.18 

3.64 

2.91 


EE3 




4.10 

3.86 

3.86 

3.90 

3.52 

3.76 

3.05 

3.81 

4.00 

4.05 

4.48 

■AM 


3.62 

2.76 

3.76 

4.19 

3.57 

3.57 

3.43 

3.10 

3.00 

2.81 

3.38 

3.48 

3.24 

3.71 

N 

3.44 





3.67 

3.78 

3.67 

3.56 





3.67 

3.78 

O 


3.62 

3.43 

3.86 

mu 


BBS 

3.43 

3.29 

3.05 



ms 

3.52 

3.62 

P 

3.84 







3.19 

2.95 

3.52 

2.38 

3.71 

ns 

3.48 

3.95 

Q 

4.13 

3.63 

3.75 

4.50 



4.13 




3.25 

4.00 

3.25 

4.38 

3.63 

AVG 

4.01 

3.74 

3.33 

4.10 

3.94 

3.79 

3.83 

3.65 

3.38 

3.45 

2.92 

3.62 

3.35 

3.65 

3.96 



m 

BXE3 

BE3 


Q20 

Q21 

Q22 

Q23 

Q24 

Q25 

Q26 

Q27 

Q28 

Q29 

Q30 

A 

BEE! 

HOT 

3.71 

BBa 



3.29 

3.81 

2.62 

3.33 

2.95 

2.05 

3.33 

3.10 

2.90 

B 

4.29 

4.19 

3.57 

3.43 

3.95 

3.43 

3.86 

4.19 

3.38 

3.52 








ms 

BEB 

Bwa 



mo 


2.95 


3.48 

2.14 

3.19 

3.14 

2.62 

KB 

mu 

BEB 

BEB 

BEB 

3.76 

3.76 

EES 

EES 

2.81 





2.95 

3.19 

E 




3.63 




4.13 

3.25 

2.88 




2.88 

3.13 

F 

§gu 

3.43 

3.33 





3.81 

3.67 

3.90 

3.71 

3.10 

3.43 

3.43 

3.43 

G 


3.38 

3.24 

3.05 

3.14 

3.33 




3.00 

2.76 

2.57 


2.76 

3.14 

H 


3.74 

3.63 


3.47 

3.47 

EfiH 

4.05 

3.68 




3.53 

3.58 

3.26 

I 

BEB 

3.67 

3.48 

BEB 

2.76 

3.43 

BEB 

4.24 

3.29 

mo 

3.19 

2.86 

3.14 

2.95 

2.62 

KB 




BEB 

3.56 

4.00 

4.00 

4.00 

3.00 


3.33 

3.78 

3.56 

3.11 

2.89 


3.27 

3.36 

3.64 

3.91 

3.09 

4.00 

3.91 




2.73 


EBB 

2.64 

3.09 

mm 

4.10 

3.90 

4.05 





4.05 

3.52 

3.48 

3.62 

3.38 

BEB 

3.05 

3.57 





3.29 



3.24 




2.86 

2.24 

2.86 

2.90 

2.86 





3.33 

BE73 


3.33 

3.89 

2.89 

3.00 

3.00 

2.56 

3.78 

2.56 

3.22 

o 

3.71 

3.00 

3.52 

2.81 

3.24 

3.38 

3.10 

3.62 

2.52 




2.81 

2.38 

3.29 

p 

3.57 

3.19 

3.95 

3.52 

tarn 


3.86 

3.67 

2.90 

3.14 

2.86 

2.95 

3.00 

2.81 

3.05 

Q 





EVvTl 

PmTy 





HEIfl 

3.75 

3.75 


3.50 

3.25 

AVG 

3.67 

3.62 

3.56 

3.46 

3.56 

3.58 

3.69 

3.92 

3.07 

3.24 

3.29 

2.68 

3.31 

3.01 

3.08 
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Sqdrn 

Q31 

Q32 

Q33 

Q34 

Q35 

Q36 



1 PE2 

Q40 

041 

042 

043 

IK9 


ISO 

3.67 

3.81 

3.57 

3.14 

3.43 

2.57 

3.71 

3.33 

3.71 

3.67 

3.57 

B 

4.29 

3.95 

4.24 

4.10 

4.05 

3.67 

3.62 

3.24 

3.86 

3.67 


IBEB 

3.62 

C 

3.81 

3.76 

3.43 

4.05 

3.76 

3.14 

2.67 

2.95 

3.62 

3.57 

MS 

EES) 

3.24 

D 

3.57 

3.62 


3.67 

3.81 

3.33 

3.14 

3.38 

3.52 

1BBSI 

1BR3 

IEEE] 

3.67 

E 

3.75 

3.63 

cvn 

EH 

Eli® 

ana 

aaii 

am 

3.63 

IBKH 

am 

EKeTSI 

3.88 

F 




ggim 

HE® 


MB 

EES 


3.67 

3.33 

3.29 

3.67 

G 



MB 

3.71 

3.43 

2.95 

3.00 

2.43 

3.24 




am 

H 

4.00 

3.79 

EH 

4.00 

3.79 

3.53 

3.53 

3.58 

3.89 



BB3 

El%] 

I 

3.33 

3.71 

3.90 







3.52 

3.86 

3.29 

am 

■■ 

4.00 

4.11 

EaD 

4.11 


ana 

4.22 

3.89 

3.89 

ana 


aaa 

MB 

m 

2.91 

3.27 

aas 

3.73 


MB 

2.73 

2.82 

3.45 

BFi® 

BUS 

am 

3.45 

WM 

4.05 

4.14 

BOB 

Eli® 


3.86 


HHI 


4.10 

3.95 

3.90 


— 



EES 










3.52 

N 

BED 


3.78 




3.33) 

3.11 

3.33 





O 

3.48 

3.48 

BEal 




ESDI 

SB! 

aaai 

3.48 

3.38 

3.10 

MB 




HEEi 

3.81 

3.71 

3.48 

PlEfill 



3.81 

3.95 

3.43 

ilSH 

Q 

3.75 

3.88 

3.88 

4.13 

3.88 

3.88 

3.75 

3.50 

3.75 

4.13 

3.88 

3.50 

3.75 

AVG 

3.66 

3.68 

3.71 

3.83 

3.71 

3.39 

3.24 

3.12 

3.65 

3.61 

3.63 

3.40 

3.62 
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APPENDIX D. 3RD MAW MAG & TYPE AIRCRAFT MOSE COMPONENTS 


MAGS 

Process 

Auditing 

Reward 

System 

Quality 

Assurance 

Risk 

Manage 

Command 
& Control 

Comms/ 
Func Rel 

MAG 

Mean 

W 

3.79 

3.58 

3.70 

3.45 

3.63 

3.57 

3.62 

X 

3.87 

3.55 

3.65 

3.46 

3.62 

3.58 

3.62 

Y 



3.65 

3.24 

3.48 

3.52 

3.53 

Z 


3.36 

3.59 

3.15 

3.45 

3.41 

3.46 

Component 

Mean 

3.82 

3.50 

3.65 

3.33 

3.55 

3.52 

3.55 


Type 

Aircraft 

Process 

Auditing 

Reward 

System 

Quality 

Assurance 

Risk 

Manage 

Command 
& Control 

Comms/ 
Func Rel 

Type 

Mean 

VMFA 

3.83 

3.69 

3.77 

3.53 

3.64 

3.62 

3.68 

VMFAT 

3.53 

3.18 

3.36 

3.13 

3.32 

3.17 

3.28 

VMGR 

3.96 

3.74 

3.89 

3.63 

3.92 

3.87 

3.83 

VMA 

3.87 

3.55 

3.65 

3.46 

3.62 

3.58 

3.62 

HMH 

3.81 

3.50 

3.68 

3.28 

3.50 

3.56 

3.55 

HMM 

3.68 

3.38 

3.51 

3.04 

3.37 

3.36 

3.39 

HMLA 

3.84 

3.33 

3.55 

3.16 

3.39 

3.40 

3.45 

HMT 

4.07 

3.53 

3.94 

3.47 

3.85 

3.66 

3.75 

Component 

Mean 

3.83 

3.49 

3.67 

3.34 

3.58 

3.53 

3.55 
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APPENDIX E. 3RD MAW COMPONENT ONE LOADINGS 













APPENDIX F. SPLUS CODE & OUTPUT FOR MCAS ANOVA & MANOVA 

> tm 

PA RS QA RM CC CF 

A 3.75 3.41 3.63 3.12 3.50 3.43 

B 4.07 3.53 3.94 3.47 3.85 3.66 

C 4.00 3.34 3.56 3.30 3.40 3.37 

F 4.02 3.67 3.60 3.65 3.68 3.55 

G 3.79 3.37 3.42 2.98 3.33 3.21 

I 3.98 3.39 3.52 3.28 3.51 3.53 

L 4.02 3.74 3.99 3.56 3.83 3.92 

M 3.61 3.25 3.37 2.99 3.17 3.19 

O 3.58 3.21 3.32 2.82 3.18 3.20 

P 3.74 3.28 3.66 3.20 3.45 3.63 

> tml_data.frame(Avg = unlist(c(tm)), Squad = dimnames(tm) 

111]][row(tm)]. Comp = dimnames(tm)[[2]][col(tm)]) 

» tmaov_anova(aov(Avg ~ Squad + Comp, data = tml)) 

> tmaov 

Analysis of Variance Table 
Response: Avg 

Terms added sequentially (first to last) 

Df Sum of Sq Mean Sq F Value Pr(F) 

Squad 9 2.27 0.252 26.7 3.22e-015 

Comp 5 2.13 0.426 45.1 2.20e-016 

Residuals 45 0.43 0.009 

> tapply(tml[,"Avg"],tml[,"Comp"],mean) 

CC CF PA QA RM RS 
3.49 3.47 3.86 3.6 3.24 3.42 

> boxplot(split(tml[,"Avg"],tml[, "Squad"] ) ) 
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> aovl_anova(aov(Avg - Squad + Comp, data = tml)) 

> aov2_aov(Avg - Squad + Comp, data = tml) 

> multl_multicomp (aov2, focus = "Comp”, method=”tukey",plot=T) 

> multi 


95 % simultaneous confidence intervals for specified 
linear combinations, by the Tukey method 

critical point: 2.976 
response variable: Squad 


intervals excluding 0 are flagged by '****' 



Estimate 

Std.Error 

Lower Bound 

Upper Bound 


o 

o 

1 

o 

'TJ 

0.021 

0.0435 

-0.1080 

0.15000 


CC-PA 

-0.366 

0.0435 

-0.4950 

-0.23700 

★ ★ 'k 

CC-QA 

-0.111 

0.0435 

-0.2400 

0.01840 


CC-RM 

0.253 

0.0435 

0.1240 

0.38200 

* * ★ * 

CC-RS 

0.071 

0.0435 

-0.0584 

0.20000 


CF-PA 

-0.387 

0.0435 

-0.5160 

-0.25800 

* * * * 

CF-QA 

-0.132 

0.0435 

-0.2610 

-0.00264 

* ★ * * 

CF-RM 

0.232 

0.0435 

0.1030 

0.36100 

* * * * 

CF-RS 

0.050 

0.0435 

-0.0794 

0.17900 


PA-QA 

0.255 

0.0435 

0.1260 

0.38400 

* * * * 

PA-RM 

0.619 

0.0435 

0.4900 

0.74800 

* * * * 

PA-RS 

0.437 

0.0435 

0.3080 

0.56600 

★ ★ * * 

QA-RM 

0.364 

0.0435 

0.2350 

0.49300 

* * * * 

QA-RS 

0.182 

0.0435 

0.0526 

0.31100 

* * * * 

RM-RS 

-0.182 

0.0435 

-0.3110 

-0.05260 

★ * * * 



-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 

simultaneous 95 % confidence limits, Tukey method 
response variable: Squad 
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> mult2_multicomp(aov2, focus = "Squad", method="tukey",plot=T) 
WARNING: Point out of bounds: x = -0.680036, y = 11.000000 

> mult2 


95 % simultaneous confidence intervals for specified 
linear combinations, by the Tukey method 

critical point: 3.3269 
response variable: Squad 


intervals excluding 0 are flagged by '****' 



Estimate 

Std.Error 

Lower Bound 

Upper Bound 


A-B 

-0.28000 

0.0561 

-0.4670 

-0.0933 

* * * * 

A-C 

-0.02170 

0.0561 

-0.2080 

0.1650 


A-F 

-0.22200 

0.0561 

-0.4080 

-0.0350 

kkkk 

A-G 

0.12300 

0.0561 

-0.0634 

0.3100 


A-1 

-0.06170 

0.0561 

-0.2480 

0.1250 


A-L 

-0.37000 

0.0561 

-0.5570 

-0.1830 


A-M 

0.21000 

0.0561 

0.0233 

0.3970 

* * * * 

A-0 

0.25500 

0.0561 

0.0683 

0.4420 

* * * * 

A-P 

-0.02000 

0.0561 

-0.2070 

0.1670 


B-C 

0.25800 

0.0561 

0.0716 

0.4450 


B-F 

0.05830 

0.0561 

-0.1280 

0.2450 


B-G 

0.40300 

0.0561 

0.2170 

0.5900 

* * ** 

B-I 

0.21800 

0.0561 

0.0316 

0.4050 


B-L 

-0.09000 

0.0561 

-0.2770 

0.0967 


B-M 

0.49000 

0.0561 

0.3030 

0.6770 

* * * * 

B-0 

0.53500 

0.0561 

0.3480 

0.7220 

* * * * 

B-P 

0.26000 

0.0561 

0.0733 

0.4470 

-k "k -k 

C-F 

-0.20000 

0.0561 

-0.3870 

-0.0133 

■k -k -k -k 

C-G 

0.14500 

0.0561 

-0.0417 

0.3320 


C-I 

-0.04000 

0.0561 

-0.2270 

0.1470 


C-L 

-0.34800 

0.0561 

-0.5350 

-0.1620 

**** 

C-M 

0.23200 

0.0561 

0.0450 

0.4180 

k 'k -k k 

C-0 

0.27700 

0.0561 

0.0900 

0.4630 

k k k k 

C-P 

0.00167 

0.0561 

-0.1850 

0.1880 


F-G 

0.34500, 

0.0561 

0.1580 

0.5320 

kkkk 

F-I 

0.16000 

0.0561 

-0.0267 

0.3470 


F-L 

-0.14800 

0.0561 

-0.3350 

0.0384 


F-M 

0.43200 

0.0561 

0.2450 

0.6180 

k k k k 

F-0 

0.47700 

0.0561 

0.2900 

0.6630 

kkkk 

F-P 

0.20200 

0.0561 

0.0150 

0.3880 

kkkk 

G-I 

-0.18500 

0.0561 

-0.3720 

0.0017 


G-L 

-0.49300 

0.0561 

-0.6800 

-0.3070 

kkkk 

G-M 

0.08670 

0.0561 

-0.1000 

0.2730 


G-0 

0.13200 

0.0561 

-0.0550 

0.3180 


G-P 

-0.14300 

0.0561 

-0.3300 

0.0434 


I-L 

-0.30800 

0.0561 

-0.4950 

-0.1220 

kkkk 

I-M 

0.27200 

0.0561 

0.0850 

0.4580 

kkkk 

1-0 

0.31700 

0.0561 

0.1300 

0.5030 

kkkk 

I-P 

0.04170 

0.0561 

-0.1450 

0.2280 


L-M 

0.58000 

0.0561 

0.3930 

0.7670 

kkkk 

L-0 

0.62500 

0.0561 

0.4380 

0.8120 

kkkk 

L-P 

0.35000 

0.0561 

0.1630 

0.5370 

kkkk 

M-0 

0.04500 

0.0561 

-0.1420 

0.2320 
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M-P “0.23000 
O-P “0.27500 


0.0561 

0.0561 


“0.4170 

-0.4620 


“0.0433 **** 
-0.0883 **** 


M-0 

M-P 

O-P 



-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 

simultaneous 95 % confidence limits, Tukey method 
response variable: Squad 


> aovl 

Analysis of Variance Table 
Response: Avg 

Terms added sequentially (first to last) 

Df Sum of Sq Mean Sq F Value Pr(F) 

Squad 9 2.27 0.252 26.7 3.22e~015 

Comp 5 2.13 0.426 45.1 2.20e-016 

Residuals 45 0.43 0.009 

> aov2 
Call: 

aov(formula = Avg - Squad + Comp, data = tml) 


Terms: 

Sum of Squares 
Deg. of Freedom 


Squad Comp Residuals 
2.27 2.13 0.43 

9 5 45 


Residual standard error: 0.0972 
Estimated effects are balanced 


> summary(aovl) 
Df 

Sum of Sq 

Mean Sq 

F Value 


Pr (F) 

Min.: 5.0 

Min.:0.425 

Min.:0.00945 

Min.:26.7 


Min.:2.22e-016 
1st Qu.: 7.0 

1st Qu.:1.280 

1st Qu.:0.13100 

1st Qu.:31.3 

1st 

Qu.:9.71e-016 
Median: 9.0 

Median:2.130 

Median:0.25200 

Median:35.9 


Median:1.72e-015 
Mean:19.7 

Mean:1.610 

Mean:0.22900 

Mean:35.9 


Mean:1.72e-015 
3rd Qu.:27.0 

3rd Qu.:2.200 

3rd Qu.:0.33900 

3rd Qu.:40.5 

3rd 

Qu.:2.47e-015 
Max.:4 5.0 

Max.:2.270 

Max.:0.42600 

Max.:45.1 


Max.:3.22e-015 

NA's:1.00e+000 



NA 1 s: 1.0 
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> summary(aov2) 


Df Sum of Sq Mean Sq 

Squad 9 2.27 0.252 

Comp 5 2.13 0.426 

Residuals 45 0.43 0.009 

> summary(anova(aov2)) 

F Value Pr(F) 

26.7 3.22e-015 
45.1 2.20e-016 



Df 

Pr (F) 

Sum ' 

of Sq 

Mean Sq 

F Value 


Min.: 5.0 
Min. : 2.226-016, 

Min. 

: 0.425 

Min.:0.00945 

Min.:26.7 


1st Qu.: 7.0 

Qu.:9.7le-016 

1st Qu. 

: 1.280 

1st Qu.:0.13100 

1st Qu.:31.3 

1st 

Median: 9.0 
Median:1.72e-015 

Median 

: 2.130 

Median:0.25200 

Median:35.9 


Mean:19.7 
Mean:1.72e-015 

Mean 

: 1.610 

Mean:0.22900 

Mean:35.9 


3rd Qu.:27.0 

Qu.:2.47e-015 

3rd Qu. 

: 2.200 

3rd Qu.:0.33900 

3rd Qu.:40.5 

3rd 

Max.:45.0 
Max.:3.22e-015 

NA ? s:1.00e+000 

Max. 

: 2.270 

Max.:0.42600 

Max.:45.1 

NA's: 1.0 
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APPENDIX G. 3 rd MAW REGRESSION DATA 


Data set = maw, Name of Fit = LI 
Normal Regression 
Kernel mean function = Identity 
Response = Incidents 

Terms = (Mean MCAS) 

Coefficient Estimates 


Label 

Estimate 



Std. Error t-value 


Constant 

-94.6783 



171.398 -0.552 


Mean MCAS 

32.7781 



48.7714 0.672 


R Squared: 



0 . 

0534434 


Sigma hat: 




29.822 


Number of cases: 



10 


Degrees of 

freedom: 



8 


Summary Analysis of 

Variance Table 


Source 

df 


SS 

MS F 

p-value 

Regression 

1 

401. 

709 

401.709 0.45 

0.5205 

Residual 

8 

7114 

.82 

889.352 


Lack of fit 7 

6993 

. 14 

999.019 8.21 

0.2626 

Pure Error 

1 

121 

.68 

121.68 



Data set = mawminusF, Name of Fit = LI 
Normal Regression 
Kernel mean function = Identity 
Response = Incidents 


Terms = (Mean 

Coefficient Estimates 
Label Estimate 

Constant 42.3433 

Mean MCAS -8.71304 

R Squared: 

Sigma hat: 

Number of cases: 
Degrees of freedom: 


Std. Error t-value 

77.0016 0.550 

22.0366 -0.395 

0.0218454 

12.8019 

9 

7 


Summary Analysis o 


Source df 

Regression 1 

Residual 7 

Lack of fit 6 

Pure Error 1 


Variance Table 

SS MS 

25.6212 25.6212 

1147.22 163.889 

1025.54 170.924 

121.68 121.68 


F p-value 

0.16 0.7043 

1.40 0.5688 
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